XML encoding scheme

ABSTRACT

Disclosed is a method ( 900 ) for communicating at least part of a structure of a document ( 104 ) described by a hierarchical representation ( 102 ). The method identifies ( 902 ) the hierarchical representation (eg. the tree structure) of the document ( 104 ). The identification is preferably performed using XML tags. The representation is then packetized ( 906 ) into a plurality of data packets. At least one link is then created ( 908 ) between a pair of the packets, the link acting to represent an interconnection between corresponding components (eg. structure and content) of the representation. The packets are then formed ( 910 ) into a stream for communication. The links maintain the hierachical representation within the packets.

This application is a National Stage Filing Under 35 U.S.C. 371 ofInternational Application No. PCT/AU01/01257, filed Oct. 5, 2001, andpublished in English as International Publication No. WO 02/29602 A1, onApr. 11, 2002.

FIELD OF THE INVENTION

The present invention relates to the encoding of XML (Extensible MarkupLanguage) documents and, in particular, to at least one of thecompression, streaming, searching and dynamic construction of XMLdocuments.

BACKGROUND

To make streaming, downloading and storing MPEG-7 descriptions moreefficient, the description can be encoded and compressed. An analysis ofa number of issues relating to the delivery of MPEG-7 descriptions hasinvolved considering the format to be used for binary encoding. Existingencoding schemes for XML, including the WBXML proposal from WAP (theWireless Application Protocol Forum), the Millau algorithm and the XMillalgorithm, have each been considered.

With WBXML, frequently used XML tags, attributes and values are assigneda fixed set of codes from a global code space. Application specific tagnames, attribute names and some attribute values that are repeatedthroughout document instances are assigned codes from some local codespaces. WBXML preserves the structure of XML documents. The content aswell as attribute values that are not defined in the Document TypeDefinition (DTD) can be stored in-line or in a string table. It isexpected that tables of the document's code spaces are known to theparticular class of applications or are transmitted with the document.

While WBXML tokenizes tags and attributes, there is no compression ofthe textual content. Whilst such is probably sufficient for the WirelessMarkup Language (WML) documents, proposed for use under the WAP, and forwhich WBXML is designed, as such documents usually have limited textualcontent, WBXML is not considered to be a very efficient encoding formatfor the typical text-laden XML documents. The Millau approach extendsthe WBXML encoding format by compressing text using a traditional textcompression algorithm. Millau also takes advantage of the schema anddatatypes to enable better compression of attribute values that are ofprimitive datatypes.

The authors of the Xmill algorithm have presented an even more complexencoding scheme, although such was not based on WBXML. Apart fromseparating structure and text encoding and using type information in DTDand schema for encoding values of built-in datatypes, that scheme also:

(i) grouped elements of the same or related types into containers (toincrease redundancy),

(ii) compressed each container separately using a different compressor,

(iii) allowed atomic compressors to be combined into more complex ones,and

(iv) allowed the use of new specialized compressors for highlyspecialized datatypes.

Nevertheless, existing encoding schemes are only designed forcompression. They do not support the streaming of XML documents. Inaddition, elements still cannot be located efficiently using theXPath/XPointer addressing scheme and a document cannot be encodedincrementally as it is being constructed.

SUMMARY OF THE INVENTION

In accordance with one aspect of the present disclosure, there isprovided a method of communicating at least part of a structure of adocument described by a hierarchical representation, said methodcomprising the steps of:

identifying said representation of said document;

packetizing said representation into a plurality of data packets, saidpackets having a predetermined size, said packetizing comprisingcreating at least one link between a pair of said packets, said linkrepresenting an interconnection between corresponding components of saidrepresentation; and

forming said data packets into a stream for communication wherein saidlinks maintain said representation within said packets.

In accordance with another aspect of the present disclosure, there isprovided a method of communicating at least part of the structure of adocument described by a hierarchical representation, said methodcomprising the steps of:

identifying at least one part of said representation and packetizingsaid parts into at least one packet of predetermined size, characterisedin that where any one or more of said parts of said representation donot fit within one said packet, defining at least one link from said onepacket to at least one further said packet into which said non-fittingparts are packetized, said link maintaining the hierarchical structureof said document in said packets.

In accordance with another aspect of the present disclosure, there isprovided a method of facilitating access to the structure of an XMLdocument, said method comprising the steps of:

identifying a hierarchical representation of said document;

packetizing said representation into a plurality of packets ofpredetermined packet size;

forming links between said packets to define those parts of saidrepresentation not able to be expressed within a packet thereby enablingreconstruction of said representations after de-packetizing.

The presently disclosed encoding and decoding schemes separate structureand text encoding and use the schema and datatypes for encoding valuesof built-in datatypes. In addition, the disclosure provides support forstreaming and allows efficient searching using XPath/XPointer-likeaddressing mechanism. Such also allows an XML document to be encoded andstreamed as it is being constructed. These features are important forbroadcasting and mobile applications. The presently disclosed encodingscheme also supports multiple namespaces and provides EBNF definitionsof the bitstream and a set of interfaces for building an extensibleencoder.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the present invention will now be describedwith reference to the drawings and Appendix, in which:

FIG. 1 schematically depicts an encoded XML document;

FIG. 2 depicts the organization of the structure segment;

FIG. 3 schematically depicts the encoder model;

FIG. 4 schematically depicts the decoder model;

FIG. 5 schematically illustrates the encoder encoding an XML documentincrementally into multiple packets;

FIGS. 6A and 6B show how node locators are used for linking a node toits sub-trees in other structure packets and how each node locatorcontains the packet number of a sub-tree's packet;

FIG. 7 schematically depicts how a long string is stored as stringfragments in multiple text packets which each packet pointing to thetext packet that contains the next fragments;

FIG. 8 is a schematic block diagram representation of a computer systemwith which the described arrangements may be implemented;

FIG. 9 is flowchart of a XML document encoding operation;

FIG. 10 is flowchart illustrating how different data types can behandled in the encoding operations; and the

Appendix provides a definition useful for the encoded bitstream and theparameters thereof.

DETAILED DESCRIPTION

The methods of encoding and decoding XML documents to be described withreference to FIGS. 1 to 7 and 9 and 10 are preferably practiced using ageneral-purpose computer system 800, such as that shown in FIG. 8wherein the processes of FIGS. 1 to 7 may be implemented as software,such as an application program executing within the computer system 800.In particular, the steps of the methods may be effected by instructionsin the software that are carried out by the computer. The software maybe divided into two separate parts; one part for carrying out theencoding/decoding methods; and another part to manage the user interfacebetween the encoding/decoding methods and the user. The software may bestored in a computer readable medium, including the storage devicesdescribed below, for example. The software is loaded into the computerfrom the computer readable medium, and then executed by the computer. Acomputer readable medium having such software or computer programrecorded on it is a computer program product. The use of the computerprogram product in the computer preferably effects an advantageousapparatus for encoding/decoding XML documents.

The computer system 800 comprises a computer module 801, input devicessuch as a keyboard 802 and mouse 803, output devices including a printer815 and a display device 814. A Modulator-Demodulator (Modem)transceiver device 816 is used by the computer module 801 forcommunicating to and from a communications network 820, for exampleconnectable via a telephone line 821 or other functional medium. Themodem 816 can be used to obtain access to the Internet, and othernetwork systems, such as a Local Area Network (LAN) or a Wide AreaNetwork (WAN). A seen, a server computer system 850 connects to thenetwork 820 enabling communications with the computer system 800. Theserver computer 850 typically has a similar structure and/or is operablein a like or complementary fashion to the computer system 800. Forexample, whilst the computer system 800 may perform an XML encodingfunction, the server computer 850 may perform a complementary XMLdecoding function, and vice versa.

The computer module 801 typically includes at least one processor unit805, a memory unit 806, for example formed from semiconductor randomaccess memory (RAM) and read only memory (ROM), input/output (I/O)interfaces including a video interface 807, and an I/O interface 813 forthe keyboard 802 and mouse 803 and optionally a joystick (notillustrated), and an interface 808 for the modem 816. A storage device809 is provided and typically includes a hard disk drive 810 and afloppy disk drive 811. A magnetic tape drive (not illustrated) may alsobe used. A CD-ROM drive 812 is typically provided as a non-volatilesource of data. The components 805 to 813 of the computer module 801,typically communicate via an interconnected bus 804 and in a mannerwhich results in a conventional mode of operation of the computer system800 known to those in the relevant art. Examples of computers on whichthe described arrangements can be practised include IBM-PC's andcompatibles, Sun Sparcstations or alike computer systems evolvedtherefrom.

Typically, the application program is resident on the hard disk drive810 and read and controlled in its execution by the processor 805.Intermediate storage of the program and any data fetched from thenetwork 820 may be accomplished using the semiconductor memory 806,possibly in concert with the hard disk drive 810. In some instances, theapplication program may be supplied to the user encoded on a CD-ROM orfloppy disk and read via the corresponding drive 812 or 811, oralternatively may be read by the user from the network 820 via the modemdevice 816. Still further, the software can also be loaded into thecomputer system 800 from other computer-readable storage media includingmagnetic tape, a ROM or integrated circuit, a magneto-optical disk, anda computer-readable card such as a PCMCIA card. The forgoing are merelyexemplary of relevant computer-readable media. Other computer-readablemedia may alternately be used. Examples of communications arrangementsby which the software may be provided to the computer module 801 includea radio or infra-red transmission channel, and the Internet andIntranets including e-mail transmissions and information recorded onWebsites and the like.

In operation the XML document encoding/decoding functions are performedon one of the server computer 850 or the computer system 800, and thepacketized bit stream so formed transmitted over the communicationsnetwork 820 for reception and decoding by the computer system 800 orserver computer 850 respectively, as the case may be. In this fashion anXML document may be conveniently communicated between two locations inan efficient manner whilst affording optimal time at the receiver todecode the document on-the-fly as it is received without a need to firstreceive the entire document.

The methods of encoding and decoding may alternatively be implemented inpart or in whole by dedicated hardware such as one or more integratedcircuits performing the functions or sub functions of encoding and/ordecoding. Such dedicated hardware may include graphic processors,digital signal processors, or one or more microprocessors and associatedmemories.

Encoding and Compressing XML

Separating Structure and Text

Traditionally, XML documents are mostly stored and transmitted in theirraw textual format. In some applications, XML documents are compressedusing some traditional text compression algorithms for storage ortransmission, and decompressed back into XML before they are parsed andprocessed.

According to the present disclosure, another way for encoding an XMLdocument is to encode the tree hierarchy of the document (such as theDOM representation of the document). The encoding may be performed in abreadth-first or depth-first manner. To make the compression anddecoding more efficient, the XML structure, denoted by tags within theXML document, can be separated from the text of the XML document andencoded. When transmitting the encoded document, the structure and thetext can be sent in separate streams or concatenated into a singlestream.

As seen in FIG. 1 and according to the instant embodiment, a treerepresentation 102 of an XML document 104, which is typically availablefrom memory, includes a number of nodes 116 and is encoded in adepth-first fashion. The structure of the document 104 and the textcontained therein can be encoded as two separate streams 106 and 108respectively as shown, or concatenated into a single stream. Thestructure stream 106 is headed by the code tables 110 and 114. Theencoded nodes 118 of the tree 102 each have a size field (notillustrated) that indicates the size of the node and includes the totalsize of its descendant nodes. Some of the encoded leaf nodes 118 containlinks 112 that link those leaf nodes to their corresponding encodedcontent in the text stream 108. Each encoded string in the text stream108 is headed by a size field (not illustrated) that indicates the sizeof the string. Where concatenated into a single stream, packetscontaining the root of the links 112 should precede those packetscontaining the text pointed to by the links 112, thereby ensuring thatthe structure component of the document 104 is received by the decoderbefore the corresponding text (content) component.

The approach shown in FIG. 1 is also depicted in FIG. 9 as a flowchartof an encoding method 900 which may be implemented as a software programrunning on the computer system 800. The method 900 communicates at leastpart of a structure of a document described by a hierarchicalrepresentation and has an entry step 902. Initially, at step 904, themethod 900 identifies the hierarchical representation (eg. the treestructure) of the document 104. The identification is preferablyperformed using the XML tags as mentioned above. With this, at step 906the representation is packetized into a plurality of data packets. Atstep 908, at least one link is created between a pair of the packets.The link acting to represent an interconnection between correspondingcomponents (eg. structure and content) of the representation. In step910, the packets are formed into a stream for communication. The linksmaintain the hierarchical representation within the packets. The method900 ends at step 912.

In general, the volume of structural information is much smaller thanthat of textual content. Structures are usually nested and repeatedwithin a document instance. Separating structure from text allows anyrepeating patterns to be more readily identified by the compressionalgorithm which, typically, examines the input stream through afixed-size window. In addition, the structure and the text streams haverather different characteristics. Hence, different and more efficientencoding methods may be applied to each of the structure and text.

The structure is critical in providing the context for interpreting thetext. Separating structure and text in an encoder allows thecorresponding decoder to parse the structure of the document morequickly thereby processing only the relevant elements while ignoringelements (and descendants) that it does not know or require. The decodermay even choose not to buffer the text associated with any irrelevantelements. Whether the decoder converts the encoded document back intoXML or not depends on the particular application to be performed (seethe discussion below on Application Program Interfaces—API's).

Code Tables

The elements of a document description and their attributes are definedin DTD's or schemas. Typically, a set of elements and their associatedattributes are repeatedly used in a document instance. Element names aswell as attribute names and values can be assigned codes to reduce thenumber of bytes required to encode them.

Typically, each application domain uses a different set of elements andtypes defined in a number of schemas and/or DTD's. In addition, eachschema or DTD may contain definitions for a different namespace. Even ifsome of the elements and types are common to multiple classes ofapplications, they are usually used in a different pattern. That is, anelement X, common to both domains A and B, may be used frequently indomain A, but rarely in domain B. In addition, existing schemas areupdated and new schemas are created all the time. Hence, it is best toleave the code assignment to organisations that overlookinteroperability in their domains. For instance, MPEG-7 descriptions areXML documents. MPEG may define the codespaces for its own descriptorsand description schemes as well as external elements and types that areused by them. MPEG may also define a method for generating codespaces.Ideally, the method should be entropy based—that is, based on the numberof occurrences of the descriptors and description schemes in adescription or a class of description (see the section on generatingcodespaces).

Separating Element and Attributes

An XML tag typically comprises an element name and a set of attributename/value pairs. Potentially, a large set of attributes can bespecified with an element instance. Hence, separating an element namefrom the attributes will allow the document tree to be parsed andelements to be located more quickly. In addition, some attributes orattribute name/value pairs tend to be used much more frequently than theothers. Grouping attribute name, value and name/value pairs intodifferent sections usually results in better compression.

Encoding Values of Built-In Datatypes and Special Types

The encoder operates to encode the values of attributes and elements ofbuilt-in (or default) datatypes into more efficient representationsaccording to their types. If the schema that contains the typeinformation is not available, the values are treated as strings. Inaddition, if a value (for instance, a single-digit integer) is moreefficiently represented as a string, the encoder may also choose totreat it as string and not to encode it. By default, strings are encodedas a Universal Text Format (UTF-8) string which provides a standard andefficient way of encoding a string of multi-byte characters. Inaddition, the UTF string includes length information avoiding theproblem of finding a suitable delimiter and allowing one to skip to theend of the string easily.

Special type encoders can be used for special data types. These specialtype encoders can be specified using the setTypeEncoder( ) interface ofthe Encoder API (as discussed below). Information about the special typeencoders is preferably stored in the header of the structure segment,advantageously as a table of type encoder identifiers. Further, thedefault type encoders (for the built-in datatypes) can be overriddenusing the same mechanism. As such where some built-in data type wouldordinarily be encoded using a default encoder, a special encoder mayalternatively be used, such necessitating identification within thebitstream that an alternative decoding process will be required forcorrect reproduction of the XML document. Each encoded value is precededby the identifier of the type encoder that was used to encode the value.

In this fashion, an XML document encoder implemented according to thepresent disclosure may include a number of encoding formats fordifferent types of structure and text within the XML document. Certainencoding formats may be built-in or default and used for well known orcommonly encountered data types. Special type encoders may be used forany special data types. In such cases, an identification of theparticular type encoder(s) used in the encoding process may beincorporated into the header of a packet, thereby enabling the decoderto identify those decoding processes required to be used for the encodedtypes in the encoded document. Where appropriate, the particular typeencoders may be accessible from a computer network via a UniformResource Indicator (URI). Where the decoder is unable to access orimplement a decoding process corresponding to an encoded typeencountered within a packet in the encoded document, a default responsemay be to ignore that encoded data, possibly resulting in thereproduction of null data (eg. a blank display). An alternative is wherethe decoder can operate to fetch the special type decoder, from aconnected network, for example using a URI that may accompany theencoded data. The URI of an encoder/decoder format may be incorporatedinto the table mentioned above and thereby included in the bitstream(see the Appendix).

In a further extension of this approach, multiple encoding formats maybe used for to a single data type. For example, text strings may beencoded differently based upon the length of the string, suchrepresenting a compromise between the time taken to perform a decodingprocess and the level of compression that may be obtained. For example,text strings with 0-9 characters may not be encoded, whereas stringswith 10-99 and 100-999 characters may be encoded with respective(different) encoding formats. Further, one or more of those encodingformats may be for a special data type. As such the encoder whenencoding text strings in this example may in practice use no encodingfor 0-9 character strings, a default encoder for 10-99 characterstrings, and a special encoder for string having more than 100 textcharacters.

FIG. 10 shows an example of a method 1000 of encoding an XML document,that has an entry point of step 1102. Initially, at step 1004, themethod 1000 examines the XML document 104 to identify each data typeforming part of the XML document 104. At step 1006, the method 1000operates to identify a first set of the data types for which acorresponding special encoding format is available. Having identifiedthe special data types, step 1008 encodes each part of the XML documenthaving a data type in the first set with the corresponding specialencoding format. Next, in step 1010, the method 1000 encodes eachremaining part of the XML document with a default encoding formatcorresponding to the data type of the remaining part. In step 1012, arepresentation is formed of the information referencing at least each ofthe data types in the first set with the corresponding special encodingformat. In step 1014, the representation is associated with the encodedparts as an encoded form of the XML document 104. The method 1000 thenends at step 1016.

The Structure Segment (or Structure Stream)

FIG. 2 shows the various sections of the structure segment (or stream)106. The structure segment begins with a header 202 and its body isdivided into a number of sections 204. The header 202 identifies theversion of the XML and that of the encoding format.

Each section 204 in the body begins with a unique signature indicatingthe section type. Hence, it is not necessary for the various sections tobe arranged in a particular order. Nevertheless, in the followingdiscussion, we assume the sections to be arranged in the order shown inFIG. 2. The section signature is followed by a size field whichindicates the size of the section.

An ID table section 206 allows elements with ID's to be located quicklyin a document hierarchy section 208. The ID table 206 may be absent froman encoded document even if the document has elements with ID's. This isbecause the DTD's or schema which contain the ID definition may not beavailable at the time of encoding.

A section 210 is preferably reserved for the document type declarationand the internal (ITD) subset. For XML Schema-based documents, forexample MPEG-7 descriptions, this section 210 will be absent.

There are sections for the code tables for namespaces 212, element names214, attribute names 216 and attribute values 218. Hereafter these codetables will be referred to as local code tables to differentiate themfrom any code tables that are pre-defined for both the encoder anddecoder and are not carried in the bitstream. For instance, there may bepre-defined code tables for MPEG-7 or XML Schema.

The local code tables are usually followed by a section containing atable of attribute name/value pairs 220 which makes use of the codesdefined in the local code tables as well as any pre-defined code tables.

The document hierarchy section 208 is the encoded tree structure of theXME document using codes from the local and the pre-defined code tables.

Apart from using code tables and type encoders for encoding, in mostcases, the encoder also compresses each section using a compressor.Instead of compressing each section of the body of the structure segment106 independently, the body of the structure segment can be compressedtogether. This may actually result in better compression ratio due tolesser overhead and the larger amount of data. However, such compressionrequires one to decompress the whole structure body in order to find outwhether a document contains a particular element. Both approaches may betested to determine which works better in practice. Nevertheless, if asection is small, compression may not be effective and the encoder maychoose not to compress the section. Each section has a compressed flagto signal whether compression has been applied. If compression has beenapplied, the size field at the beginning of each section indicates thecompressed (rather than the uncompressed) size of the section in bytes.

Potentially, a different compressor can be used for each section takinginto account the characteristics of the data in each section.Information about the compressors used is provided in the header. Thedefault is to use ZLIB for compressing all the sections in the structuresegment as well as the text segment. The ZLIB algorithm generates aheader and a checksum that allow the integrity of the compressed data tobe verified at the decoder end.

The Text Segment (or Text Stream)

The text segment 108 begins with a text segment signature followed by asize field that indicates the size of the encoded text. The text segmentcontains a sequence of UTF-8 strings which are the text of the elements.

The Encoder and Decoder Models

The Encoder Model

FIG. 3 shows an XML encoder model 300 incorporating an encoder 302 forencoding the XML document 104 into a bitstream 306 for storage ortransmission. The encoder model 300 may be implemented as a softwareprogram or sub-programs operating within the computer module 801, theprogram being typically stored in the HDD 810 and read and controlled inits execution by the processor 805. The bitstream 306 may be transmittedupon creation via the I/O interface 808 and network 820 forcomplementary decoding and reproduction by the server computer 850.Alternatively, the bitstream 306 may be stored in the HDD 810 or as aCD-ROM in the drive 812 for subsequent reproduction. The encoder 302 maysupport an Application Program Interface (API) 308 (eg. the DOM API) sothat the document tree 102 can be encoded as the tree 102 is beingcreated. A standard library 310 (for XML) is used to provide code tables312, encoders 314 for built-in datatypes, and default compressors 316that may be used in the encoding processes. Domain-specific libraries318 may also be defined for various domains. Each domain-specificlibrary 318 may contain code tables 320 for the particular domain andencoders 322 for some data types. An application can also providespecific modules 324 including application-specific encoders 326 forspecial data types as discussed above and corresponding compressors 328.However, these type encoders 326 and compressors 328 have to be eitherdownloadable and platform-independent or preinstalled at the decoderend. An application can also instruct the encoder 326 to use itspre-defined code tables 330. The code tables 330 can be incorporatedinto the bitstream 306 or pre-installed at the decoder end. Each of theindividual encoders and compressors shown in FIG. 3 may be implementedby software (sub)programs or, in some instances special purpose hardware(eg. for fast encoding).

The Decoder Model

FIG. 4 shows a complementary XML decoder model 400 including a decoder402 for decoding the XML bitstream 306 to output an XML document 104.Alternatively, the decoder may support an API 408 (eg. the SAX (“simpleAPI for XML”) or DOM API) that allows an application to construct itsown internal model of the document tree 102. This saves the decoder 402from outputting the XML document 104 and the application from re-parsingthe reconstructed XML document 104. In either case, the decoder 402 usesthe standard library 410, any domain-specific libraries 418 as well asany pre-installed or downloaded application-specific modules 424 (thatwere used by the encoder) when decoding the XML bitstream 306. In FIG.4, elements of the decoder model 400 are numbered in a similar fashionto that of FIG. 3, such that where a difference of 100 exists in thenumbering, the elements have corresponding like functions. The decodermodel 400 may for example be implemented within the computer module 801to decode the bitstream 306 received via the network 820 from the servercomputer 850. Alternatively, the decoder model 400 may operate to decodea bitstream obtained from the CD-ROM, for example. Like the encoder 302,software and hardware decoding processes may be used within the decoder402.

In most cases, the decoder 402 at the client end need not validate thedecoded XML document 104 of FIG. 4 against their DTD's or schemas.Validation at the client side is costly, inefficient and most likelyredundant. The decoder 104 may assume that the XML documents have beenvalidated against their DTD's or schemas at the server end. Similarly,the underlying transport as well as any error detection mechanism suchas checksums that is built into the binary format should be capable ofcatching any transmission error.

Locating Elements

XML elements can be referenced and located using ID's or XPath/XPointerfragments. As mentioned earlier, the ID table 206 of the structuresegment 106 allows elements with ID's to be located quickly in thedocument hierarchy section 208. Any text and attributes associated withan element can then be located efficiently using the locators in theencoded elements.

Below are some examples of XPath fragments that can be appended to anUniform Resource Indicator (URI):

-   -   /doc/chapter[2]/section[3]        -   selects the third section of the second chapter of doc    -   chapter[contains(string(title),“Overview”)]        -   selects the chapter children of the context node that have            one or more title children containing the text “Overview”    -   child::*[self::appendix or self::index]        -   selects the appendix and index children of the context node    -   child::*[self::chapter or self::appendix] [position( )=last( )]        -   selects the last chapter or appendix child of the context            node    -   para[@type=“warning”]        -   selects all para children of the context node that have a            type attribute with value “warning”    -   para[@id]        -   selects all the para children of the context node that have            an id attribute.

An XPath/XPointer fragment consists of a list of location stepsrepresenting the absolute or relative location of the requiredelement(s) within an XML document. Typically, the fragment contains alist of element names. Predicates and functions may be used, as in theexamples above, to specify additional selection criteria such as theindex of an element within an array, the presence of an attribute,matching attribute value and matching textual content.

The compactness of the encoded document hierarchy allows it to be parsed(and instantiated) without expanding into a full object treerepresentation. The fragment address is first translated into an encodedform. One of the consequences of such a translation process is that itallows one to determine immediately whether the required element(s)actually occurred in the document. Matching the components of theencoded fragment address is also much more efficient than matchingsub-strings. The design allows simple XPath/XPointer fragments (whichare most frequently used) to be evaluated quickly. Searching thedocument hierarchy first also greatly narrows the scope of subsequentevaluation steps in the case of a more complex fragment address.

Packetizing the Bitstream for Streaming

Streaming XML

Traditionally, XML documents are mostly stored and transmitted in theirraw textual format. In some applications, XML documents are compressedusing some traditional text compression algorithms for storage ortransmission, and decompressed back into XML before they are parsed andprocessed. Although compression may greatly reduce the size of an XMLdocument, under such circumstances an application still must receive theentire XML document before parsing and processing can be performed.

Streaming an XML document implies that parsing and processing can startas soon as sufficient portion of the XML document is received. Suchcapability will be most useful in the case of a low bandwidthcommunication link and/or a device with very limited resources.

Because an ordinary XML parser expects an XML document to be well-formed(ie. having matching and non-overlapping start-tag and end-tag pairs),the parser can only parse the XML document tree in a depth-first mannerand cannot skip parts of the document unless the content of the XMLdocument is reorganized to support it.

Packetizing the Bitstream

Encoding an XML document into a complete structure segment 106 and acomplete text segment 108 as described earlier will greatly reduce thesize of the data and, at the same time, allow some transmission error tobe detected. Nevertheless, the decoder 402 still has to receive a largeamount of the encoded data before it can process it. For instance, thedecoder 402 will have received the code tables 110 in their entiretybefore parsing of the document hierarchy can commence. At the same time,the decoder 402 has to wait for the arrival of certain segment of thetext segment 108 to get the text that is associated with a node. Toallow processing to be started as soon as possible at the decoder end,the XML document 104, as seen in FIG. 5, has to be encoded incrementallyallowing small packets 502 of encoded data 500 to be sent to the decoder402 as they become available. In FIG. 5, the packets 504 denotestructure packets and the packets 506 denote text packets. These packetsare preceded by a header packet 508 and followed by a trailer packet510. In the preferred arrangement, each data packet 502 has the samestructure as a complete structure segment 106 or a complete text segment108. At the same time, each packet 502 may be dependent on those packets502 sent before it or, in some implementations, on a predeterminednumber of packets sent after it. Such a predetermine number may bedetermined dynamically.

Apart from the need for processing a document while it was beingdelivered, an encoder/decoder typically has an output/input buffer offixed size. Accordingly, except for very short documents, the encoder302 has to encode an XML document incrementally into multiple packets.Each of the packets 502 (including 504, 506, 508 and 510) is headed by apacket header. The packet header contains a packet number that is usedas a packet ID as well as for ordering the packets and detecting anymissing packets. The packet header also contains a size field whichindicates the size of the packet 502 in bytes and a type field whichindicate whether the packet is a structure packet 504, a text packet506, a header packet 508, a trailer packet 510 or a further type ofpacket 502, named a command packet, not illustrated in FIG. 5, butdescribed later in this document.

For each structure packet 504, the ID table incorporated thereincontains only the ID's of those elements included in the packet. Itscode tables contain only new codes that have not been transmitted. Codesthat have been transmitted will not be re-assigned or remapped. Thedefault implementation simply appends new value to the table and usesthe index (augmented by the base index of the table) of the entries astheir codes. A slightly more complicated (but more code efficient)method is to count the number of occurrences of the values and remap thecodes so that values that occur more frequently are remapped to shortercodes just before the packets are output. If a pre-defined code table isused or if the remapping is not based on the number of occurrences,sorting the values before compressing may result in better compressionrate. A different algorithm for assigning code can be implemented.Nevertheless, once output, the codes are fixed and cannot be re-assignedto other values or re-mapped in subsequent packets. Pre-defined codetables can also be specified using the UseCodeTable( ) method of theEncoder Interface described later in this specification. The method alsoallows one to specify whether the pre-defined code table is to beencoded with the data into the bitstream. The code tables of a number ofnamespaces which are fundamental to XML (or an application domain suchas MPEG-7) are expected to be hardwired to all XML (MPEG-7) encoders anddecoders and need not be encoded into the bitstream.

If an ID, an element name, an attribute name, or an attribute value islonger than a pre-defined length, it will be encoded in a text packetand a string locator rather than the actual string will appear in thetables.

The document hierarchy section of a structure packet contains a sequenceof nodes. Each node has a size field that indicates its (encoded) sizein bytes including the total size of its descendant nodes encoded in thepacket. The node can be an element node, a comment node, a text node ora node locator. Each node has a nodeType field that indicates its type.

The document hierarchy may contain:

(i) a complete document tree: this is only possible for very shortdocument;

(ii) a complete sub-tree: the sub-tree is the child of another nodeencoded in an earlier packet; and

(iii) an incomplete sub-tree: the sub-tree is incomplete because thewhole sub-tree cannot be encoded into one packet due to time and/or sizeconstraints.

Node locators are used in the manner shown in FIG. 6A, for a treestructure 622 which has incomplete sub-trees 602 and 604, for locatingthe missing nodes and the descendants of the incomplete sub-trees. Inthis regard, and with reference to the earlier example, whilst thehierarchical tree-representation 102 of the document 104 is known whenencoding takes place, upon decoding of the communicated packets, onlyportions of the tree representation 102 will typically be madeavailable. As more packets are received the tree may be reconstructed.For example, in the data stream shown in FIG. 6B, a packet 620 (beingthe #2 packet in the data stream in this example) includes part of thetree structure 622 of a document, that structure including nodes A, B1,B2 and B3. However, in this example, the size of the packet 620 isinsufficient to describe the entire tree structure 622 and toaccommodate other nodes, such as B4 and D1. Node locators 608 and 606respectively are thus incorporated into the descriptions of thecorresponding parent nodes (B3 and B2 respectively) and contain therespective packet numbers 610 and 612 of a structure packets thatcontains a sequence of missing nodes and their sub-trees. As such, onreceiving the sequence of packets illustrated in FIG. 6B, part of thetree 622 can be reconstructed upon receiving the packet (#2) 620 and thebranch including node D1 can be reconstructed upon receiving packet (#7)610 and the balance of the tree reconstructed upon receiving packet(#20) 612.

Each element node preferably contains a namespace code, an element(name) code, and, if the element has attributes, the byte offset of thefirst attribute in the attribute name/value pair table and the number ofattributes.

Each text node or comment node typically contains a text locator ratherthan the actual text. The text locator specifies the packet number of atext packet and a byte offset into the text packet.

In some cases, a string may exceed the maximum size of a packet. Wheresuch occurs, the string is stored as fragments over multiple textpackets, as shown in FIG. 7. Each text packet 702 has a flag 704indicating whether it contains a list of UTF-8 encoded strings andstring locators or a string fragment. In the case of a string fragment,the packet number of the next fragment is also included. If a textpacket contains the last (or the only) fragments of a string, the packetnumber for the next fragment is set to zero, as shown.

Commands for Constructing Document Tree

An XML document may be packetized for streaming to the receiver as it isbeing encoded or even generated (according to some pre-defined DTD orschema). In this case, the XML document is typically constructed inreal-time using an API such as a DOM APL Instead of parsing an XML file,the encoder 302 operates to construct the bit stream 306 from the memoryrepresentation directly. Nodes and sub-trees inserted and appended usingthe API are encoded as (binary) command packets to modify the memoryrepresentation at the decoder end. The packet number ensures that thecommand packets are executed in the correct sequence.

Since the nodes transmitted are parts of the same document (thatconforms to some pre-defined DTD or schema) and the document is on-lineand in-sync between the encoder 302 and decoder 402 all the time, thereshould not be any consistency issue in relation to the content of thenodes. In some presentations, certain information has only temporalrelevance. That is, some information is only relevant within a certainperiod of time during the presentation. Information units (for example,the score of a football match) that are relevant to two different timeinstances of the presentation may themselves be inconsistent. Apresentation description scheme is desirable to establish the timing andsynchronization model of a presentation. The timing of any media objectincluding XML data can be indicated by a start time and a duration. Sucha presentation encoder/decoder pair would typically include an XMLencoder/decoder as described above arranged internally. The presentationdecoder, rather than the XML decoder, operates to interpret the starttime and duration attributes. The presentation encoder also decideswhether or not to remove from memory an XML sub-tree that is no longerrelevant. As long as the XML encoder/decoder is concerned, there is noconsistency issue. If the generator is always required to generate validdocument (fragments), then there is no need for a command to remove(possibly inconsistent or invalid) nodes or sub-trees. That is, onlyinsert and append commands are needed.

A command packet contains the path of (the root of) the sub-tree to beappended or inserted and the packet number of the structure packet thatcontains the sub-tree. For example, returning to FIG. 6B, if the locator608 for node B4 was not able to be accommodated in the packet 620, thena command packet would have to be inserted between packets #2 and #20that effectively attaches node B4 to node A. That command packet wouldthen include a locator pointing to the packet 612 including thestructure defined by node B4.

The Definition of the Bitstream

The bitstream 306 is preferably defined in Extended Backus-Naur Form(ENBF) in the fashion defined by the Appendix. Characters are enclosedby single quote and strings by double quotes. Unless stated otherwise,UCS characters in UTF-8 encoding and UTF strings (that include lengthinformation) are assumed.

API

API for Documents and Schemas

It is not always necessary for the decoder 402 to convert an encodeddocument back into XML. As indicated above, the decoder 402 may supportan API such as the SAX APL the DOM API, or other proprietary APL toallow an application to access the decoded content directly. This savesthe decoder 402 from having to reconstruct and output the XML documentand the application from having to re-parse the reconstructed XMLdocument.

An application may also have to access information stored in schemas. Asschemas are also XML documents, they can be encoded in the same way.Using existing SAX or DOM API for accessing and interpreting schemadefinitions is extremely tedious. A parser that supports a schema API,such as the Schema API defined in Wan E., Anderson M., Lennon A.,Description Object Model (DesOM). Doc. ISO/IEC JTC1/SC29/WG11MPEG00/M5817, Noordwijkerhout, March 2000, will make accessing thedefinitions of schemas much easier.

To allow the values of built-in datatypes and special types to beencoded efficiently, an encoder has to be able to obtain typeinformation from the schemas. Hence, a schema API is also extremelyimportant to the encoder 302.

API for Encoders

The binary format proposed below allows for the implementation ofencoders of various capabilities and complexity. The interfacesdescribed in this section allow one to construct a basic encoder thatcan be extended to provide the more complicated features supported bythe encoding scheme.

Encoder Interface

-   void SetMaxPacketSize(in unsigned long maxPacketSize)    -   Set the maximum packet size in bytes.-   void SetMaxPrivateDataSize(in unsigned long maxPrivateDataSize)    -   Set the maximum size of the private data in byte. Note that the        amount of private data that can be included in a packet is        limited by the maximum size of the packet. A large amount of        private data is not expected as such works against the objective        of reducing the size of the bitstream.-   void SetHeaderUserData(in ByteArray headerData)    -   Write the user data to the header packet. Any existing data will        be overwritten.-   void UseCodeTable(in CodeTable codeTable, in Boolean encodeIt)    -   Inform the encoder of a pre-defined code table and whether the        code table should be encoded with the data.-   void SetCompressor(in Section section, in Inflater compressor)    -   Instruct the encoder to use the specified compressor for the        specified section. Section is an enumeration with the following        values: STRUCT_BODY=1, TEXT_BODY=2, ID_TABLE=3, NS_SECT=4,        ELEMENT_SECT=5, ATTR_NAME_SECT=6, ATTR_VALUE_SECT=7,        ATTR_PAIR_SECT=8, DOC_HIERARCHY_SECT=9. Inflater has the same        interface as Inflater of the java.util.zip package.-   void Flush( )    -   Flush the packets in the buffer to the output stream.-   void OnOutput( )    -   Receive notification before the set of packets in the buffer is        output to allow the application to insert application        specific-data to the packets.-   void SetPacketUserData(in ByteArray userData)    -   Write the user data to each of the packets except any header        packet in the buffer. Any existing user data will be        overwritten.        Code Table Interface-   unsigned short GetSize( )    -   Get the number of entries in the code table.-   wstring GetNamespace(in unsigned short i)    -   Get the namespace of the value associated with the ith entry of        the code table.-   wstring GetValue(in unsigned short i)    -   Get the value associated with the ith entry of the code table.-   wstring GetType(in unsigned short i)    -   Get the type of the value associated with the ith entry of the        code table.-   ByteArray GetCode(in unsigned short i)    -   Get the code associated with the ith entry of the code table.-   unsigned short GetIndexByCode(in ByteArray code)    -   Get the value associated with a code.-   unsigned short GetIndexByValue(in wstring value)    -   Get the value associated with a code.-   unsigned short GetMaxCodeValue( )    -   Get the maximum code value reserved by the code table. The        encoder is free to use code value above the maximum code value.        Depending on application, an encoder may also be implemented to        use holes left by a pre-defined code table.        Type Encoder Interface-   ByteArray Encode(in wstring text)    -   Encode the value into a byte array given its text        representation.-   wstring Decode(in ByteArray encodedText)    -   Decode an encoded value into the text representation of the        value.        Encoding the XML Data, in Particular MPEG-7 Descriptions of a        Presentation

If (fragments of) XML data including MPEG-7 descriptions (which are XMLdata used for describing audio-visual (AV) content) are to be streamedand presented with AV content, the timing of and the sychronizationbetween the media objects (including the XML data) have to be specified.Like XML, the DDL (the description definition language of XML) does notdefine a timing and synchronization model for presenting media objects.As mentioned above, a SMIL-like MPEG-7 description scheme called hereinPresentation Description Scheme is desired to provide the timing andsynchronization model for authoring multimedia presentations.

It has been suggested that MPEG-7 descriptions can be treated in thesame way as AV objects. This means that each MPEG-7 descriptionfragment, like AV objects, used in a presentation will be tagged with astart time and a duration defining its temporal scope. This allows bothMPEG-7 fragments and AV objects to be mapped to a class of media objectelements of the Presentation Description Scheme and subjected to thesame timing and sychronization model. Specifically, in the case of aSMIL-based Presentation Description Scheme, a new media object elementsuch as an <mpeg7> tag can be defined. Alternately, MPEG-7 descriptionscan also be treated as a specific type of text.

It is possible to send different types of MPEG-7 descriptions in asingle stream or in separate streams. It is also possible to send anMPEG-7 description fragment that has sub-fragments of different temporalscopes in a single data stream or in separate streams. This is a rolefor the presentation encoder, in contrast to the XML encoder 300discussed earlier.

The presentation encoder wraps an XML packet with a start time and aduration signalling when and for how long the content of the packet isrequired or relevant. The packet may contain:

(i) multiple short description fragments (each with their own temporalscope) concatenated together to achieve high compression rate andminimize overhead;

(ii) a single description fragment; and

(iii) part of a large description fragment.

In the case where the packet contains multiple description fragments,the start time of the packet is the earliest of the start times of thefragments while the duration of the packet is the difference between thelatest of the end time of the fragments (calculated by adding theduration of the fragment to its start time) and the start time of thepacket.

In broadcasting applications, to enable users to tune into thepresentation at any time, relevant materials have to be repeated atregular interval. While only some of the XML packets have to be resentas some of the XML packets sent earlier may no longer be relevant, theheader packet needs to be repeated. This means that, in the case ofbroadcasting applications, the header packet may be interspersed amongstructure, text and command packets to reset the transmission to a knownstate.

INDUSTRIAL APPLICABILITY

It is apparent from the above that the arrangements described areapplicable to the computer and data processing industries and to theefficient use of communication resources associated therewith whilstaffording the ability to work with partially received information.

The foregoing describes only one or more embodiments of the presentinvention, and modifications and/or changes can be made thereto withoutdeparting from the scope and spirit of the invention, the embodiment(s)being illustrative and not restrictive. For example, whilst describedwith reference to XML documents, the procedures disclose herein areapplicable to any hierarchical representation, such as a treerepresentation of a document.

APPENDIX Definition of the Bitstream The bitstream will be defined inExtended Backus-Naur Form (ENBE). Character will be enclosed by singlequote and string by double quote. Unless stated otherwise, UCScharacters in UTF-8 encoding and UTF strings (that include lengthinformation) are assumed. xmlbitStream ::= xml_packet+         N.B.: Thebitstream of an encoded XML document consists of a sequence of packets.The sequence begins with a header packet and ends with a trailer packet.Packet xml_packet ::= packet_header packet_body packet_header ::=packet_signature   packet_number   packet_size   packet_typepacket_private_data packet_number ::= variable_length_natural_number        N.B.: packet_number has to be greater than 0. packet_type ::=header_packet | structure_packet | text_packet | trailer_packet |command_packet packet_signature ::= ‘x’ ‘m’ ‘l’ ‘b’ ‘i’ ‘n’ ‘p’ ‘k’packet_size ::= unsigned_short         N.B.: With unsigned_short, anunsigned integer in the range 0 − 65535 is represented using 2 byteswith the first byte being the hihg-order byte of the integer.packet_private_data ::= byte_array packet_body ::= header_body |trailer_body | structure_body | text_body | command_body header_packet::= ‘h’ structure_packet ::= ‘s’ text_packet ::= ‘t’ trailer_packet ::=‘e’ command_packet ::= ‘c’ byte_array ::= size_in_byte byte*size_in_byte ::= variable_length_natural_number         N.B.: Withvariable_length_natural_number, a natural number in the range 0 −1,073,741,823 is represented using 1 to 4 bytes with the first bytebeing the high-order byte of the number. The two most significant bitsof the high-order byte is actually used to indicate the number ofadditional bytes used for representing the number. For instance, ‘01’implies one additional byte or a 2-byte representation and ‘11’ implies3 additional bytes or a 4-byte representation.) Byte ::= [ #x00  #xFF ]Header header_body ::= enoding_version  xml_version  xml_params max_packet_size max_decompressed_packet_size          max_packet_numbersection_compressor_list  type_encoder_list xml_params ::= countxml_encoding xml_standalone encoding_version ::= “1.0” xml_version ::=“1.0” count ::= variable_length_natural_number xml_encoding ::=UTF8_string         N.B.: With UTF8_String, the first two bytes is anunsigned short, the UTF length, that specifies the number of additionalbytes to be read. The additional bytes contain the UTF-8 encoding of thestring. xml_standalone ::= ‘y’ | ‘n’ max_packet_size ::=variable_length_natural_number         N.B.: A value of zero impliesthat the maximum packet size is unknown. max_packet_number ::=variable_length_natural_number         N.B.:  A value of zero impliesthat the maximum number of packets is unknown. section_compressor_list::= count ( section_ID compressor_URI )* type_encoder_list ::= count (type_ID type_encoder_URI )* compressor_URI ::= URI type_encoder_URI ::=URI URI ::= UTF8_string section_ID ::= struct_body_ID | text_body_ID |id_table_ID | ns_section_ID | element_sect_ID | attribute_name_sect_ID |attribute_value_sect_ID | attribute_pair_sect_ID | doc_hierarchy_sect_IDstruct_body_ID ::= ‘s’ text_body_ID ::= ‘t’ id_table_ID ::= ‘i’ns_section_ID ::= ‘n’ element_sect_ID ::= ‘e’ attribute_name_sect_ID ::=‘a’ attribute_value_sect_ID ::= ‘v’ attribute_pair_sect_ID ::= ‘p’doc_hierarchy_sect_ID ::= ‘d’ type_ID ::= [ #x00  #xFF ] other_type_ID::= #x00 string_ID ::= #x01 string_locator_ID ::= #x02 boolean_ID ::=#x03 byte_ID ::= #x04 unsigned_short_ID ::= #x05 short_ID ::= #x06unsigned_long_ID ::= #x07 long_ID ::= #x08 float_ID ::= #x09 double_ID::= #x0A date_ID ::= #x0B time_ID ::= #x0C         N.B.: The above listfor built-in datatypes are not complete. Type 00-0F are for built-indatatypes. An XML encoder can assign type 10-FF to application-specifictypes. The application is responsible for providing the (Java) typeencoder and decoder for any application- specific types. These typeencoder and decoder must be pre-installed or downloaded before they arerequired. When type information is not available, XML text and attributevalues will be treated as string. Trailer trailer_body ::=         N.B.:At the moment, the trailer packet is only used to signal the end of theXML document. The body of the trailer packet is emptly. Structure Packetstructure_body ::= [ ID_table_section ] [ internal_subset_section ] [ns_table_section ] [ element_name_codetable_section ] [attribute_name_codetable_section ]  [ attribute_value_codetable_section] [ attribute_name_value_pair_section ] [ document_hierarchy_section ]        N.B.: Although the above EBNF rule defines the various sectionsof the body of a structure packet to be arranged in a particular order,the sections are actually allowed to be arranged in any order as eachsection is identified by its unique signature. ID Table SectionID_table_section ::= ID_table_section_signature    section_size   compressed entry_count ( ID_table | compressed_ID_table )section_size ::= size_in_byte         N.B.: section_size stores the sizeof the section excluding its signature. compressed ::= boolean        N.B.: The compressed flag indicates whether the table iscompressed.         N.B.: With boolean, a byte value of 1 representstrue an a byte value of 0 represents false. entry_count ::=variable_length_natural_number size_of_compressed_ID_table ::=variable_length_natural_number ID_table ::= ( ID_stringoffset_to_the_document_hierarchy )*         N.B.: ID_table defines thestructure of the uncompressed ID table. The ID table only collects ID ofnodes (not including nodes referred to by node locators) that appears inthe document hierarchy of the same packet. If type information is notavailable during encoding, IDs will not be collected into the ID tableeven if they are present in the document as there is no way the encodercan identify them. ID_string ::= UTF8_stringoffset_to_the_document_hierarchy ::= byte_offset         N.B.:   offset_to_the_document_hierarchy is the byte offset todocument_hierarchy     in     the   (uncompressed)document_hierarchy_section not the byte offset to the (uncompressed)document_hierarhy_section byte_offset ::= variable_length_natural_numberID_table_section_signature ::= #xFF01 Internal Subset SectionInternal_subset_section ::= internal_subset_section_signature      section_size compressed [ byte* ]         N.B.: The detail of theinternal subset section has yet to be defined.NS_table_section_signature ::= #xFF02 Namespace Table SectionNS_table_section ::= NS_table_section signature     section_size    compressed entry_count index_base ( NS_table | compressed_NS_table )index_base ::= variable_length_natural_number         N.B.: The indexinto the NS_table is used as the namespace code. The base of the indexis specified in the field index_base. The namespace code 0 is reservedfor the null namespace. Hence, a namespace table cannot have anindex_base of 0. NS_table ::= (NS_URI )*         N.B.: NS_table definesthe structure of the uncompressed NS table. The index into the table isused as the namespace code. The base of the index is specified in thefield index_base. The namespace code 0 is reserved for the nullnamespace. Hence, a namespace table cannot have an index_base of 0.NS_URI ::= URI NS_table_section_signature ::= #xFF03 Cod Tabl Sectionselement_name_codetable_section::=  element_name_codetable_section_signature section_size    compressed   entry_count    index_base ( element_name_codetable |compressed_element_name_codetable ) attribute_name_codetable_section::=  attribute_name_codetable_section_signature section_size   compressed    entry_count    index_base ( attribute_name_codetable |compressed_attribute_name_codetable ) attribute_value_codetable_section::=  attribute_value_codetable_section_signature section_size   compressed    entry_count    index_base has_predefined_code     (    attribute_value_codetable    | compressed_attribute_value_codetable)         N.B.: The index into each code table is used as the codeunless there is a predefined code. The code tables allow the mappingbetween the codes used for the encoding and the actual values. The baseof the index for each table is specified in the field index_base of thattable. Only positive codes are allowed. Hence, index_base cannot have avalue of zero. element_name_codetable_section_signature ::= #xFF04attribute_name_codetable_section_signature ::= #xFF05attribute_value_codetable_section_signature ::= #xFF06has_predefined_code ::= boolean         N.B.: The has_predefined_codeflag specify whether the code table has a predefined_code column.Element name code table element_name_codetable ::=element_name_code_table entry*         N.B.: element_name_codetabledefines the structure of the uncompressed element name code table. Theindex into the table is used as the element name code unless there is apredefined code. The base of the index is specified in the fieldindex_base. The code 0 is reserved. Hence, a code table cannot have anindex_base of 0. element_name_codetable_entry   ::= ns_code   element_name    type_ID [ predefined_code ]         N.B.: Except forthe built-in datatypes and special types that are known to the encoder,textual content of all other type will be encoded as string.predefined_code ::= byte_array         N.B.: An empty predefined_codeimplies that there is no predefined code for that entry. This should nothappen. If an value is missing from a pre-defined code table. Theencoder has to generate a code for the value and store it in thepredefined_code field. element_name ::= non_empty_UTF8_string | ( #x0000string_locator )         N.B.: The element names are usually storedin-line in the table. However, if an element name is too long, it can bestored in a separate text packet and a string locator is used in thetable instead. string_locator ::= text_packet_number byte_offset        N.B.: A byte_offset specifies the offset into the text packet'sbody where the string can be found. non_empty_UTF8_string ::=UTF8_string −“ ” Attribute name code table attribute_name_codetable ::=attribute_name_code_table_entry*         N.B.: attribute_name_codetabledefines the structure of the uncompressed attribute name code table. Theindex into the table is used as the attribute name code unless there isa predefined code. The base of the index is specified in the fieldindex_base. The code 0 is reserved. Hence, a code table cannot have anindex_base of 0. attribute_name_codetable_entry   ::=  ns_code   attribute_name    type_ID [ predefined_code ]         N.B.: Exceptfor the built-in datatypes and special types that are known to theencoder, textual content of all other type will be encoded as string.attribute_name ::= non_empty_UTF8_string | ( #x0000 string_locator )        N.B.: The attribute names are usually stored in-line in thetable. However, if an attribute name is too long, it can be stored in aseparate text packet and a string locator is used in the table instead.Attribute value code table attribute_value_codetable ::=attribute_value_code_table_entry*         N.B.:attribute_value_codetable defines the structure of the uncompressedattribute value code table. The index into the table is used as theattribute value code unless there is a predefined code. The base of theindex is specified in the field index_base. The code 0 is reserved.Hence, a code table cannot have an index_base of 0.attribute_value_codetable_entry ::=  ns_code    attribute_value   type_ID [ predefined_code ]         N.B.: Except for the built-indatatypes and special types that are known to the encoder, textualcontent of all other type will be encoded as string. attribute_value ::=encoded_value         N.B.: The attribute value are usually storedin-line in the table. encoded_value ::= encoded_value_of_non_string_type| non_empty_UTF8_string | ( “ ” #x00 ) | ( #x000 string_locator )        N.B.: Values are encoded according to their types. Except forbuilt-in datatypes and special types that are known to the encoder,value are encoded as string.         N.B.: An empty UTF8-string has tobe followed by #x00 to distinguish it from a valid string locator.Again, if an attribute name is too long, it can be stored in a separatetext packet and a string locator is used in the table instead. AttributeName/Value Pair Section attribute_name_value_pair_section::=  attribute_name_value_pair_section_signature section_size   compressed    entry_count    index_base (attribute_name_value_pair_table                |compressed_attribute_name_value_pair_table )attribute_name_value_pair_table = attribute_name_value_pair_entry*        N.B.: attribute_name_value_pair_table defines the structure ofthe uncompressed attribute name/value pair table. The base of the index(>0) is specified in the field index_base.attribute_name_value_pair_entry ::=attribute_name_codeattribute_value_code attribute_name_value_pair_section_signature    ::=#xFF07 Document Hi rarchy S ction document_hierarchy_section::=document_hierarchy_section_signature   section_size compressed ( subtree| compressed_subtree ) subtree ::= node         N.B.: subtree definesthe structure of the uncompressed XML sub-tree. node ::= node_size node_type    ( element_node  | text_node  | comment_node | node_locator)         N.B.: The node_size includes the size of the node and itsdescendent nodes encoded in the same packet. node_type::=   (element_node_signature      element_flag)     | (  ( text_node_signature  | comment_node_signature      |locator_node_signature ) #x0 ) element_node_signature  ::= #x3text_node_signature  ::= #x5 comment_node_signature ::= #x9locatore_node_signature  ::= #xC element_flag ::= has_attributes |has_children | has_attributes_and_children has_attributes ::= 0x1has_children ::= 0x2 has_attributes_and_children ::= 0x3 element_node::= element_name_code [ attributes ] [ child_node* ] child_nodes ::=node attributes ::=        index_of_starting_attribute_name value_pairnumber_of_attributes number_of_attributes ::=variable_length_natural_number text_locator ::= string_locatorcomment_node ::= text_locator node_locator ::= packet_number Text Packettext_body ::= compressed (encoded_text | compressed_encoded_text )encoded_text ::= (0x00 encoded_value* ) | (next_packet_numberUTF8_string )         N.B.: If next_packet_number is zero, the firststring of the text packet may be the last fragments of a long string, Ifnext_packet_number is non- zero, the whole text packet contains a singlefragment of a string. next_packet_number variable length natural_numberCommand Packet command_body ::= command path packet_number_of_subtree        N.B.: The subtree to be added is defined in the structure packetwith the specified packet number. command ::= insert_command | append_command insert_command ::= #x01 append_command ::= #x02 path::= URI_reference URI-reference ::= UTF8_string

1. A method, which is performed by an apparatus that includes aprocessor coupled to a memory, of transmitting a portion of a firsthierarchical representation of a structured document located at asender, the first hierarchical representation including structurecontent and textual content, to modify a second hierarchicalrepresentation located at a receiver, the method comprising steps of:creating a plurality of content packets containing a portion of thetextual content and at least one structure packet containing a portionof the structure content; creating at least one structure link from theat least one structure packet to at least one further structure packetinto which further parts of the structure are packetized, the at leastone structure link maintaining a hierarchical structure of the documentin the structure packets; generating, for a sub-tree of the firsthierarchical representation, a content link to corresponding textualcontent located in one of the plurality of content packets and storingthe content link and the at least one structure link as a part of thestructure content; constructing a command packet to modify the secondhierarchical representation located at the receiver, the command packetidentifying a path of a root node of the sub-tree of the firsthierarchical representation to be appended to the second hierarchicalrepresentation and a packet number included in the at least onestructure packet, wherein the packet number identifies the at least onestructure packet, and wherein the identified at least one structurepacket and the at least one further structure packet contain thesub-tree; transmitting the plurality of content packets, the at leastone structure packet, the at least one further structure packet, and thecommand packet from the sender to the receiver; identifying, at thereceiver, one content packet of the plurality of content packets, theone content packet containing the portion of the textual content thatcorresponds to the sub-tree of the first hierarchical representation,using the generated content link; and modifying, at the receiver, a partof the second hierarchical representation in accordance with the onecontent packet and the command packet.
 2. A method according to claim 1,wherein the first hierarchical representation comprises a treerepresentation.
 3. A method according to claim 1, wherein the documentcomprises an XML document.
 4. A method according to claim 1, wherein thegenerating step comprises creating at least one link between at leastone structure packet containing a node of the structure and at least onetextual content packet containing at least one of: an element name, anattribute name, and an attribute value of the node.
 5. A methodaccording to claim 1, wherein the document is an XML document, saidmethod further comprising steps of: examining the XML document toidentify each data type forming part of the XML document; identifying afirst set of data types for which a corresponding special encodingformat is available; first encoding each part of the XML document havinga data type in the first set with the corresponding special encodingformat; second encoding each remaining part of the XML document with adefault encoding format corresponding to the data type of the remainingpart; forming a representation of information referencing at least eachdata type in the first set with the corresponding special encodingformat; and associating the representation and the encoded parts as anencoded form of the XML document.
 6. A method according to claim 5,wherein said encoding separately encodes structure parts and contentparts of the XML document, and the formed representation is retained ina header portion of the encoded form of the XML document.
 7. A methodaccording to claim 5 or 6, wherein the formed representation is retainedin the header portion as a table.
 8. A method according to claim 5,wherein said first encoding comprises examining a data type of the firstset and a corresponding part of the XML document and determining one ofthe encoding formats to be applied to the corresponding part.
 9. Amethod according to claim 5, wherein at least said second encodingcomprises selecting one of a plurality of the encoding formatscorresponding to a data type of the remaining part and encoding theremaining part with the selected encoding format.
 10. A method, which isperformed by an apparatus that includes a processor coupled to a memory,of transmitting a portion of a first hierarchical representation of astructured document, the first hierarchical representation includingstructure content and textual content to modify a second hierarchicalrepresentation located at a receiver, the method comprising steps of:creating a plurality of content packets containing a portion of thetextual content and at least one structure packet containing a portionof the structure content; creating at least one structure link from theat least one structure packet to at least one further structure packetinto which further parts of the structure are packetized, the at leastone structure link maintaining a hierarchical structure of the documentin the structure packets; generating, for a sub-tree of the firsthierarchical representation, a content link to corresponding textualcontent located in one of the plurality of content packets and storingthe content link and the at least one structure link as a part of thestructure content, the content link enabling identification of which ofthe transmitted plurality of content packets contains the correspondingtextual content; constructing a command packet to modify the secondhierarchical representation located at the receiver, the command packetidentifying a path of a root node of the sub-tree of the firsthierarchical representation to be appended to the second hierarchicalrepresentation and a packet number included in the at least onestructure packet. wherein the packet number identifies the at least onestructure packet, and wherein the identified at least one structurepacket and the at least one further structure packet contain thesub-tree; and transmitting the plurality of content packets, the atleast one structure packet, the at least one further structure packet,and the command packet from the apparatus to the receiver to allow thereceiver to modify a part of the second hierarchical representation inaccordance with the identified content packet and the command packet.11. A method, which is performed by a receiver apparatus that includes aprocessor coupled to a memory, of receiving a portion of a firsthierarchical representation of a structured document, the firsthierarchical representation including structure content and textualcontent to modify a second hierarchical representation, the methodcomprising steps of: receiving a plurality of content packets containinga portion of the textual content, at least one structure packetcontaining a portion of the structure content, at least one furtherstructure packet and at least one structure link from the at least onestructure packet to the at least one further structure packet into whichfurther parts of the structure are packetized, the at least onestructure link maintaining a hierarchical structure of the document inthe structure packets, the structure content having, for a sub-tree ofthe first hierarchical representation, a content link to correspondingtextual content located in one of the plurality of content packets;receiving a command packet to modify the second hierarchicalrepresentation located at the receiver apparatus, the command packetidentifying a root node of the sub-tree of the first hierarchicalrepresentation to be appended to the second hierarchical representationand a packet number included in the at least one structure packet,wherein the packet number identifies the at least one structure packet,the identified structure packet, and the at least one further structurepacket containing the sub-tree; identifying one content packet of theplurality of content packets, the one content packet containing theportion of the textual content that corresponds to the sub-tree of thefirst hierarchical representation, using the content link; and modifyinga part of the second hierarchical representation in accordance with theone content packet and the command packet.
 12. A computer-readablestorage medium having a computer program recorded thereon, the programbeing executable by computer apparatus to communicate a portion of afirst hierarchical representation of a structured document to areceiver, the first hierarchical representation including structurecontent and textual content to modify a second hierarchicalrepresentation located at the receiver, the program comprising steps of:creating a plurality of content packets containing a portion of thetextual content and at least one structure packet containing a portionof the structure content; creating at least one structure link from theat least one structure packet to at least one further structure packetinto which further parts of the structure are packetized, the at leastone structure link maintaining a hierarchical structure of the documentin the structure packets; generating, for a sub-tree of the firsthierarchical representation, a content link to corresponding textualcontent located in one of the plurality of content packets and storingthe content link and the at least one structure link as a part of thestructure content, the content link enabling identification of which ofthe transmitted plurality of content packets contains the correspondingtextual content; constructing a command packet to modify the secondhierarchical representation located at the receiver, the command packetidentifying a path of a root node of the sub-tree of the firsthierarchical representation to be appended to the second hierarchicalrepresentation and a packet number included in the at least onestructure packet, wherein the packet number identifies the at least onestructure packet, and wherein the identified at least one structurepacket and the at least one further structure packet contain thesub-tree; and transmitting the plurality of content packets, the atleast one structure packet, the at least one further structure packet,and the command packet from the computer apparatus to the receiver toallow the receiver to modify a part of the second hierarchicalrepresentation in accordance with the one content packet and the commandpacket.
 13. The computer-readable storage medium according to claim 12,said program further comprising a step for: creating at least onestructure link from the at least one structure packet to at least onefurther structure packet into which further parts of the structure arepacketized, the at least one structure link maintaining the hierarchicalstructure of the document in the structure packets.
 14. Thecomputer-readable storage medium according to claim 12, wherein thehierarchical representation comprises a tree representation.
 15. Thecomputer-readable storage medium according to claim 12, wherein thedocument comprises an XML document.
 16. The computer-readable storagemedium according to claim 12, said program further comprising a step forcreating at least one link between at least one structure packetcontaining a node of the structure and at least one textual contentpacket containing at least one of: an element name, an attribute name,and an attribute value of the node.
 17. The computer-readable storagemedium according to claim 12, wherein the document is an XML document,said program further comprising steps for: examining the XML document toidentify each data type forming part of the XML document; identifying afirst set of data types for which a corresponding special encodingformat is available; first encoding each part of the XML document havinga data type in the first set with the corresponding special encodingformat; second encoding each remaining part of the XML document with adefault encoding format corresponding to the data type of the remainingpart; forming a representation of information referencing at least eachdata type in the first set with the corresponding special encodingformat; and associating the representation and the encoded parts as anencoded form of the XML document.
 18. The computer-readable storagemedium according to claim 17, wherein said encoding separately encodesstructure and content parts of the XML document, and the formedrepresentation is retained in a header portion of the encoded form ofsaid XML document.
 19. The computer-readable storage medium according toclaim 17 or 18, wherein the formed representation is retained in theheader portion as a table.
 20. The computer-readable storage mediumaccording to claim 17, wherein said first encoding comprises examining adata type of the first set and a corresponding part of the XML documentand determining one of the encoding formats to be applied to thecorresponding part.
 21. The computer-readable storage medium accordingto claim 17, wherein at least said second encoding comprises selectingone of a plurality of the encoding formats corresponding to a data typeof the remaining part and encoding the remaining part with the selectedencoding format.
 22. An apparatus, which includes a processor coupled toa memory, for communicating a portion of a first hierarchicalrepresentation of a structured document, the first hierarchicalrepresentation including structure content and textual content, tomodify a second hierarchical representation located at a receiver, theapparatus comprising: a first creating unit, which creates a pluralityof content packets containing a portion of the textual content and atleast one structure packet containing a portion of the structurecontent; a second creating unit, which creates at least one structurelink from the at least one structure packet to at least one furtherstructure packet into which further parts of the structure arepacketized. the at least one structure link maintaining a hierarchicalstructure of the document in the structure packets; a generating unit,which generates, for a sub-tree of the first hierarchicalrepresentation, a content link to corresponding textual content locatedin one of the plurality of content packets and storing the content linkand the at least one structure link as a part of the structure content,the content link enabling identification of which of the transmittedplurality of content packets contains the corresponding textual content;a constructing unit, which constructs a command packet to modify thesecond hierarchical representation located at the receiver, the commandpacket identifying the a root node of the sub-tree of the firsthierarchical representation to be appended to the second hierarchicalrepresentation and a packet number included in the at least onestructure packet, wherein the packet number identifies the at least onestructure packet, and wherein the identified at least one structurepacket and the at least one further structure packet contain thesub-tree; and a transmission unit, which transmits the plurality ofcontent packets, the at least one structure packet, the at least onefurther structure packet, and the command packet, from the apparatus, tothe receiver to allow the receiver to modify a part of the secondhierarchical representation in accordance with the one content packetand the command packet.
 23. An apparatus according to claim 22, whereinthe document is an XML document, said apparatus further comprising: anexamining unit, which examines the XML document to identify each datatype forming part of the XML document; an identifying unit, whichidentifies a first set of data types for which a corresponding specialencoding format is available; a first encoding unit, which encodes eachpart of the XML document having a data type in the first set with thecorresponding special encoding format; a second encoding unit, whichencodes each remaining part of the XML document with a default encodingformat corresponding to the data type of remaining part; a forming unit,which forms a representation of information referencing at least eachdata type in the first set with the corresponding special encodingformat; and an associating unit, which associates the representation andthe encoded parts as an encoded form of the XML document.